6.6 Orthogonal Projections and the Spectral Theorem
In this section, we rely heavily on Theorems 6.16 (p. 369) and 6.17 (p. 371) to develop an elegant representation of a normal (if $F=C$) or a selfadjoint (if $F=R$) operator T on a finitedimensional inner product space. We prove that T can be written in the form ${\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$, where ${\lambda}_{1},\text{}{\lambda}_{2},\text{}\dots ,\text{}{\lambda}_{k}$ are the distinct eigenvalues of T ${\text{T}}_{1},{\text{T}}_{2},\text{}\dots ,{\text{T}}_{k}$ are orthogonal projections. We must first develop some results about these special projections.
We assume that the reader is familiar with the results about direct sums developed at the end of Section 5.2. The special case where V is a direct sum of two subspaces is considered in the exercises of Section 1.3.
Recall from the exercises of Section 2.1 that if $\text{V}={\text{W}}_{1}\oplus {\text{W}}_{2}$, then a linear operator T on V is the projection on ${\text{W}}_{1}$ along ${\text{W}}_{2}$ if, whenever $x={x}_{1}+{x}_{2}$, with ${x}_{1}\in {\text{W}}_{1}$ and ${x}_{2}\in {\text{W}}_{2}$, we have $\text{T}(x)={x}_{1}$. By Exercise 27 of Section 2.1, we have
So $\text{V}=\text{R}(\text{T})\oplus \text{N}(\text{T})$. Thus there is no ambiguity if we refer to T as a “projection on ${\text{W}}_{1}$” or simply as a “projection.” In fact, it can be shown (see Exercise 17 of Section 2.3) that T is a projection if and only if $\text{T}={\text{T}}^{2}$. Because $\text{V}={\text{W}}_{1}\oplus {\text{W}}_{2}={\text{W}}_{1}\oplus {\text{W}}_{3}$ does not imply that ${\text{W}}_{2}={\text{W}}_{3}$, we see that ${\text{W}}_{1}$ does not uniquely determine T. For an orthogonal projection T, however, T is uniquely determined by its range.
Definition.
Let V be an inner product space, and let $\text{T}:\text{V}\to \text{V}$ be a projection. We say that T is an orthogonal projection if $\text{R}{(\text{T})}^{\perp}=\text{N}(\text{T})$ and $\text{N}{(\text{T})}^{\perp}=\text{R}(\text{T})$.
Note that by Exercise 13(c) of Section 6.2, if V is finitedimensional, we need only assume that one of the equalities in this definition holds. For example, if $\text{R}{(\text{T})}^{\perp}=\text{N}(\text{T})$, then $\text{R}(\text{T})=\text{R}{(\text{T})}^{\perp \perp}=\text{N}{(\text{T})}^{\perp}$.
An orthogonal projection is not the same as an orthogonal operator. In Figure 6.5, T is an orthogonal projection, but T is clearly not an orthogonal operator because $\text{T}(v)\ne v$.
Now assume that W is a finitedimensional subspace of an inner product space V. In the notation of Theorem 6.6 (p. 347), we can define a function $\text{T}:\text{V}\to \text{V}$ by $\text{T}(y)=u$. It is easy to show that T is an orthogonal projection on W. We can say even more—there exists exactly one orthogonal projection on W. For if T and U are orthogonal projections on W, then $\text{R}(\text{T})=\text{W}=\text{R}(\text{U})$. Hence $\text{N}(\text{T})=\text{R}{(\text{T})}^{\perp}=\text{R}{(\text{U})}^{\perp}=\text{N}(\text{U})$, and since every projection is uniquely determined by its range and null space, we have $\text{T}=\text{U}$. We call T the orthogonal projection of V on W.
To understand the geometric difference between an arbitrary projection on W and the orthogonal projection on W, let $\text{V}={\text{R}}^{2}$ and $\text{W}=\text{span}\{(1,\text{}1)\}$. Define U and T as in Figure 6.5, where T(v) is the foot of a perpendicular from v on the line $y=x$ and $\text{U}({a}_{1},\text{}{a}_{2})=({a}_{1},\text{}{a}_{1})$. Then T is the orthogonal projection of V on W, and U is a different projection on W. Note that $v\text{T}(v)\in {\text{W}}^{\perp}$, whereas $v\text{U}(v)\notin {\text{W}}^{\perp}$. From Figure 6.5, we see that T(v) is the “best approximation in W to v”; that is, if $w\in \text{W}$, then $wv\ge \text{T}(v)v$. In fact, this approximation property characterizes T. These results follow immediately from the corollary to Theorem 6.6 (p. 348).
As an application to Fourier analysis, recall the inner product space H and the orthonormal set S in Example 9 of Section 6.1. Define a trigonometric polynomial of degree n to be a function $g\in \text{H}$ of the form
where ${a}_{n}$ or ${a}_{n}$ is nonzero.
Let $f\in \text{H}$. We show that the best approximation to f by a trigonometric polynomial of degree less than or equal to n is the trigonometric polynomial whose coefficients are the Fourier coefficients of f relative to the orthonormal set S. For this result, let $\text{W}=\text{span}\left(\right\{{f}_{j}:\text{}j\le n\left\}\right)$, and let T be the orthogonal projection of H on W. The corollary to Theorem 6.6 (p. 348) tells us that the best approximation to f by a function in W is
For an application of this material to electronic music, visit goo.gl/
An algebraic characterization of orthogonal projections follows in the next theorem.
Theorem 6.24.
Let V be an inner product space, and let T be a linear operator on V. Then T is an orthogonal projection if and only if T has an adjoint T* and ${\text{T}}^{2}=\text{T}=\text{T*}$.
Proof.
Suppose that T is an orthogonal projection. Since ${\text{T}}^{2}=\text{T}$ because T is a projection, we need only show that T* exists and $\text{T}=\text{T*}$. Now $\text{V}=\text{R}(\text{T})\oplus \text{N}(\text{T})$ and $\text{R}{(\text{T})}^{\perp}=\text{N}(\text{T})$. Let $x,\text{}y\in \text{V}$. Then we can write $x={x}_{1}+{x}_{2}$ and $y={y}_{1}+{y}_{2}$, where ${x}_{1},\text{}{y}_{1}\in \text{R}(\text{T})$ and ${x}_{2},\text{}{y}_{2}\in \text{N}(\text{T})$. Hence
and
So $\langle x,\text{T}(y)\rangle =\langle \text{T}(x),\text{}y\rangle $ for all $x,\text{}y\in \text{V}$ thus T* exists and $\text{T}=\text{T}*$.
Now suppose that ${\text{T}}^{2}=\text{T}=\text{T*}$. Then T is a projection by Exercise 17 of Section 2.3, and hence we must show that $\text{R}(\text{T})=\text{N}{(\text{T})}^{\perp}$ and $\text{R}{(\text{T})}^{\perp}=\text{N}(\text{T})$. Let $x\in \text{R}(\text{T})$ and $y\in \text{N}(\text{T})$. Then $x=\text{T}(x)=\text{T*}(x)$, and so
Therefore $x\in \text{N}{(\text{T})}^{\perp}$, from which it follows that $\text{R}(\text{T})\subseteq \text{N}{(\text{T})}^{\perp}$.
Let $y\in \text{N}{(\text{T})}^{\perp}$. We must show that $y\in \text{R}(\text{T})$, that is, $\text{T}(y)=y$. Now
Since $y\text{T}(y)\in \text{N}(\text{T})$, the first term must equal zero. But also
Thus $y\text{T}(y)=0$; that is, $y=\text{T}(y)\in \text{R}(\text{T})$. Hence $\text{R}(\text{T})=\text{N}{(\text{T})}^{\perp}$.
Using the preceding results, we have $\text{R}{(\text{T})}^{\perp}=\text{N}{(\text{T})}^{\perp \perp}\supseteq \text{N}(\text{T})$ by Exercise 13(b) of Section 6.2. Now suppose that $x\in \text{R}{(\text{T})}^{\perp}$. For any $y\in \text{V}$, we have $\langle \text{T}(x),\text{}y\rangle =\langle x,\text{T*}(y)\rangle =\langle x,\text{T}(y)\rangle =0$. So $\text{T}(x)=0$, and thus $x\in \text{N}(\text{T})$. Hence $\text{R}{(\text{T})}^{\perp}=\text{N}(\text{T})$.
Let V be a finitedimensional inner product space, W be a subspace of V, and T be the orthogonal projection of V on W. We may choose an orthonormal basis $\beta =\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ for V such that $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$ is a basis for W. Then ${[\text{T}]}_{\beta}$ is a diagonal matrix with ones as the first k diagonal entries and zeros elsewhere. In fact, ${[\text{T}]}_{\beta}$ has the form
If U is any projection on W, we may choose a basis $\gamma $ for V such that ${[\text{U}]}_{\gamma}$ has the form above; however $\gamma $ is not necessarily orthonormal.
We are now ready for the principal theorem of this section.
Theorem 6.25 (The Spectral Theorem).
Suppose that T is a linear operator on a finitedimensional inner product space V over F with the distinct eigenvalues ${\lambda}_{1},\text{}{\lambda}_{2},\text{}\dots ,\text{}{\lambda}_{k}$. Assume that T is normal if $F=C$ and that T is selfadjoint if $F=R$. For each $i(1\le i\le k)$, let ${\text{W}}_{i}$ be the eigenspace of T corresponding to the eigenvalue ${\lambda}_{i}$, and let ${\text{T}}_{i}$ be the orthogonal projection of V on ${\text{W}}_{i}$. Then the following statements are true.

(a) $\text{V}={\text{W}}_{1}\oplus {\text{W}}_{2}\oplus \cdots \oplus {\text{W}}_{k}\text{.}$

(b) If ${\text{W}}_{i}^{\prime}$ denotes the direct sum of the subspaces ${\text{W}}_{j}$ for $j\ne i$, then ${\text{W}}_{i}^{\perp}={\text{W}}_{i}^{\prime}$.

(c) ${\text{T}}_{i}{\text{T}}_{j}={\delta}_{ij}{\text{T}}_{i}$ for $1\le i,\text{}j\le k\text{.}$

(d) $\text{I}={\text{T}}_{1}+{\text{T}}_{2}+\cdots +{\text{T}}_{k}.$

(e) $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}.$
Proof.
(a) By Theorems 6.16 (p. 369) and 6.17 (p. 371), T is diagonalizable;
so
by Theorem 5.10 (p. 277).
(b) If $x\in {\text{W}}_{i}$ and $y\in {\text{W}}_{j}$ for some $i\ne j$, then $\langle x,\text{}y\rangle =0$ by Theorem 6.15(d) (p. 368). It follows easily from this result that ${\text{W}}_{i}^{\prime}\subseteq {\text{W}}_{i}^{\perp}$. From (a), we have
On the other hand, we have $\mathrm{dim}({\text{W}}_{i}^{\perp})=\mathrm{dim}(\text{V})\mathrm{dim}({\text{W}}_{i})$ by Theorem 6.7(c) (p. 349). Hence ${\text{W}}_{i}^{\prime}={\text{W}}_{i}^{\perp}$, proving (b).
(c) The proof of (c) is left as an exercise.
(d) Since ${\text{T}}_{i}$ is the orthogonal projection of V on ${\text{W}}_{i}$, it follows from (b) that $\text{N}({\text{T}}_{i})=\text{R}{({\text{T}}_{i})}^{\perp}={\text{W}}_{i}^{\perp}={\text{W}}_{i}^{\prime}$. Hence, for $x\in \text{V}$, we have $x={x}_{1}+{x}_{2}+\cdots +{x}_{k}$, where ${\text{T}}_{i}(x)={x}_{i}\in {\text{W}}_{i}$, proving (d).
(e) For $x\in \text{V}$, write $x={x}_{1}+{x}_{2}+\cdots +{x}_{k}$, where ${x}_{i}\in {\text{W}}_{i}$. Then
The set $\{{\lambda}_{1},\text{}{\lambda}_{2},\text{}\dots ,\text{}{\lambda}_{k}\}$ of eigenvalues of T is called the spectrum of T, the sum $\text{I}={\text{T}}_{1}+{\text{T}}_{2}+\cdots +{\text{T}}_{k}$ in (d) is called the resolution of the identity operator induced by T, and the sum $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ in (e) is called the spectral decomposition of T. The spectral decomposition of T is unique up to the order of its eigenvalues.
With the preceding notation, let $\beta $ be the union of orthonormal bases of the ${\text{W}}_{i}$’s and let ${m}_{i}=\mathrm{dim}({\text{W}}_{i})$. (Thus ${m}_{i}$ is the multiplicity of ${\lambda}_{i}$.) Then ${[\text{T}]}_{\beta}$ has the form
that is, ${[\text{T}]}_{\beta}$ is a diagonal matrix in which the diagonal entries are the eigenvalues ${\lambda}_{i}$ of T, and each ${\lambda}_{i}$ is repeated ${m}_{i}$ times. If ${\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ is the spectral decomposition of T, then it follows (from Exercise 7) that $g(\text{T})=g({\lambda}_{1}){\text{T}}_{1}+g({\lambda}_{2}){\text{T}}_{2}+\cdots +g({\lambda}_{k}){\text{T}}_{k}$ for any polynomial g. This fact is used below.
We now list several interesting corollaries of the spectral theorem; many more results are found in the exercises. For what follows, we assume that T is a linear operator on a finitedimensional inner product space V over F.
Corollary 1.
If $F=C$ then T is normal if and only if $\text{T*}=g(\text{T})$ for some polynomial g.
Proof.
Suppose first that T is normal. Let $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ be the spectral decomposition of T. Taking the adjoint of both sides of the preceding equation, we have $\text{T*}={\overline{\lambda}}_{1}{\text{T}}_{1}+{\overline{\lambda}}_{2}{\text{T}}_{2}+\cdots +{\overline{\lambda}}_{k}{\text{T}}_{k}$ since each ${\text{T}}_{i}$ is selfadjoint. Using the Lagrange interpolation formula (see page 53), we may choose a polynomial g such that $g({\lambda}_{i})={\overline{\lambda}}_{i}$ for $1\le i\le k$. Then
Conversely, if $\text{T*}=g(\text{T})$ for some polynomial g, then T commutes with T* since T commutes with every polynomial in T. So T is normal.
Corollary 2.
If $F=C$, then T is unitary if and only if T is normal and $\lambda =1$ for every eigenvalue $\lambda $ of T.
Proof.
If T is unitary, then T is normal and every eigenvalue of T has absolute value 1 by Corollary 2 to Theorem 6.18 (p. 379).
Let $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ be the spectral decomposition of T. If $\lambda =1$ for every eigenvalue $\lambda $ of T, then by (c) of the spectral theorem,
Hence T is unitary.
Corollary 3.
If $F=C$, then T is selfadjoint if and only if T is normal and every eigenvalue of T is real.
Proof.
Suppose that T is normal and that its eigenvalues are real. Let $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ be the spectral decomposition of T. Then
Hence T is selfadjoint.
Now suppose that T is selfadjoint and hence normal. That its eigenvalues are real has been proved in the lemma to Theorem 6.17 (p. 371).
Corollary 4.
Let T be as in the spectral theorem with spectral decomposition $\text{T}={\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$. Then each ${\text{T}}_{j}$ is a polynomial in T.
Proof.
Choose a polynomial ${g}_{j}(1\le j\le k)$ such that ${g}_{j}({\lambda}_{i})={\delta}_{ij}$. Then
Exercises

Label the following statements as true or false. Assume that the underlying inner product spaces are finitedimensional.

(a) All projections are selfadjoint.

(b) An orthogonal projection is uniquely determined by its range.

(c) Every selfadjoint operator is a linear combination of orthogonal projections.

(d) If T is a projection on W, then T(x) is the vector in W that is closest to x.

(e) Every orthogonal projection is a unitary operator.


Let $\text{V}={\text{R}}^{2},\text{W}=\text{span}(\{(1,\text{}2)\})$, and $\beta $ be the standard ordered basis for V. Compute ${[\text{T}]}_{\beta}$, where T is the orthogonal projection of V on W. Do the same for $\text{V}={\text{R}}^{3}$ and $\text{W}=\text{span}(\{(1,\text{}0,\text{}1)\})$.

For each of the matrices A in Exercise 2 of Section 6.5:

(1) Verify that ${\text{L}}_{A}$ possesses a spectral decomposition.

(2) For each eigenvalue of ${\text{L}}_{A}$, explicitly define the orthogonal projection on the corresponding eigenspace.

(3) Verify your results using the spectral theorem.


Let W be a finitedimensional subspace of an inner product space V. Show that if T is the orthogonal projection of V on W, then $\text{I}\text{T}$ is the orthogonal projection of V on ${\text{W}}^{\perp}$.

Let T be a linear operator on a finitedimensional inner product space V.

(a) If T is an orthogonal projection, prove that $\text{T}(x)\le x$ for all $x\in \text{V}$. Give an example of a projection for which this inequality does not hold. What can be concluded about a projection for which the inequality is actually an equality for all $x\in \text{V}$?

(b) Suppose that T is a projection such that $\text{T}(x)\le x$ for all $x\in \text{V}$. Prove that T is an orthogonal projection.


Let T be a normal operator on a finitedimensional inner product space. Prove that if T is a projection, then T is also an orthogonal projection.

Let T be a normal operator on a finitedimensional complex inner product space V. Use the spectral decomposition ${\lambda}_{1}{\text{T}}_{1}+{\lambda}_{2}{\text{T}}_{2}+\cdots +{\lambda}_{k}{\text{T}}_{k}$ of T to prove the following results.

(a) If g is a polynomial, then
$$g(\text{T})={\displaystyle \sum _{i=1}^{k}g({\lambda}_{i}){T}_{i}.}$$ 
(b) If ${\text{T}}^{n}={\text{T}}_{0}$ for some n, then $\text{T}={\text{T}}_{0}$.

(c) Let U be a linear operator on V. Then U commutes with T if and only if U commutes with each ${\text{T}}_{i}$.

(d) There exists a normal operator U on V such that ${\text{U}}^{2}=\text{T}$.

(e) T is invertible if and only if ${\lambda}_{i}\ne 0$ for $1\le i\le k$.

(f) T is a projection if and only if every eigenvalue of T is 1 or 0.

(g) $\text{T}=\text{T*}$ if and only if every ${\lambda}_{i}$ is an imaginary number.


Use Corollary 1 of the spectral theorem to show that if T is a normal operator on a complex finitedimensional inner product space and U is a linear operator that commutes with T, then U commutes with T*.

Referring to Exercise 20 of Section 6.5, prove the following facts about a partial isometry U.

(a) U*U is an orthogonal projection on W.

(b) $\text{UU*U}=\text{U.}$


Simultaneous diagonalization. Let U and T be normal operators on a finitedimensional complex inner product space V such that $\text{TU}=\text{UT}$. Prove that there exists an orthonormal basis for V consisting of vectors that are eigenvectors of both T and U. Hint: Use the hint of Exercise 14 of Section 6.4 along with Exercise 8.

Prove (c) of the spectral theorem. Visit goo.gl/
utQ9Pb for a solution.