6.2 The GramSchmidt Orthogonalization Process and Orthogonal Complements
In previous chapters, we have seen the special role of the standard ordered bases for ${\text{C}}^{n}$ and ${\text{R}}^{n}$. The special properties of these bases stem from the fact that the basis vectors form an orthonormal set. Just as bases are the building blocks of vector spaces, bases that are also orthonormal sets are the building blocks of inner product spaces. We now name such bases.
Definition.
Let V be an inner product space. A subset of V is an orthonormal basis for V if it is an ordered basis that is orthonormal.
Example 1
The standard ordered basis for ${\text{F}}^{n}$ is an orthonormal basis for ${\text{F}}^{n}$.
Example 2
The set
is an orthonormal basis for ${\text{R}}^{2}$.
The next theorem and its corollaries illustrate why orthonormal sets and, in particular, orthonormal bases are so important.
Theorem 6.3.
Let V be an inner product space and $S=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$ be an orthogonal subset of V consisting of nonzero vectors. If $y\in \text{span}(S)$, then
Proof.
Write $y={\displaystyle \sum _{i=1}^{k}{a}_{i}{v}_{i}}$, where ${a}_{1},\text{}{a}_{2},\text{}\dots ,\text{}{a}_{k}\in F$. Then, for $1\le j\le k$,
we have
So ${a}_{j}={\displaystyle \frac{\langle y,\text{}{v}_{j}\rangle}{{v}_{j}{}^{2}}}$, and the result follows.
The next corollary follows immediately from Theorem 6.3.
Corollary 1.
If, in addition to the hypotheses of Theorem 6.3, S is orthonormal and $y\in \text{span}(S)$, then
If V possesses a finite orthonormal basis, then Corollary 1 allows us to compute the coefficients in a linear combination very easily. (See Example 3.)
Corollary 2.
Let V be an inner product space, and let S be an orthogonal subset of V consisting of nonzero vectors. Then S is linearly independent.
Proof.
Suppose that ${v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\in S$ and
As in the proof of Theorem 6.3 with $y=0$, we have ${a}_{j}=\langle 0,\text{}{v}_{j}\rangle /{v}_{j}{}^{2}=0$ for all j. So S is linearly independent.
Example 3
By Corollary 2, the orthonormal set
obtained in Example 8 of Section 6.1 is an orthonormal basis for ${\text{R}}^{3}$. Let $x=(2,\text{}1,\text{}3)$. The coefficients given by Corollary 1 to Theorem 6.3 that express x as a linear combination of the basis vectors are
and
As a check, we have
Corollary 2 tells us that the vector space H in Section 6.1 contains an infinite linearly independent set, and hence H is not a finitedimensional vector space.
Of course, we have not yet shown that every finitedimensional inner product space possesses an orthonormal basis. The next theorem takes us most of the way in obtaining this result. It tells us how to construct an orthogonal set from a linearly independent set of vectors in such a way that both sets generate the same subspace.
Before stating this theorem, let us consider a simple case. Suppose that $\{{w}_{1},\text{}{w}_{2}\}$ is a linearly independent subset of an inner product space (and hence a basis for some twodimensional subspace). We want to construct an orthogonal set from $\{{w}_{1},\text{}{w}_{2}\}$ that spans the same subspace. Figure 6.1 suggests that the set $\{{v}_{1},\text{}{v}_{2}\}$, where ${v}_{1}={w}_{1}$ and ${v}_{2}={w}_{2}c{w}_{1}$, has this property if c is chosen so that ${v}_{2}$ is orthogonal to ${w}_{1}$.
To find c, we need only solve the following equation:
So
Thus
The next theorem shows us that this process can be extended to any finite linearly independent subset.
Theorem 6.4.
Let V be an inner product space and $S=\{{w}_{1},\text{}{w}_{2},\text{}\dots ,\text{}{w}_{n}\}$ be a linearly independent subset of V. Define ${S}^{\prime}=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$, where ${v}_{1}={w}_{1}$ and
Then ${S}^{\prime}$ is an orthogonal set of nonzero vectors such that $\text{span}({S}^{\prime})=\text{span}(S)$.
Proof.
The proof is by mathematical induction on n, the number of vectors in S. For $k=1,\text{}2,\text{}\dots ,\text{}n,$ let ${S}_{k}=\{{w}_{1},\text{}{w}_{2},\text{}\dots ,\text{}{w}_{k}\}$. If $n=1$, then the theorem is proved by taking ${S}_{1}^{\prime}={S}_{1};$ i.e., ${v}_{1}={w}_{1}\ne 0$. Assume then that the set ${S}_{k1}^{\prime}=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k1}\}$ with the desired properties has been constructed by the repeated use of (1). We show that the set ${S}_{k}^{\prime}=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k1},\text{}{v}_{k}\}$ also has the desired properties, where ${v}_{k}$ is obtained from ${S}_{k1}^{\prime}$ by (1). If ${v}_{k}=0$, then (1) implies that ${w}_{k}\in \text{span}({S}_{k1}^{\prime})=\text{span}({S}_{k1})$, which contradicts the assumption that ${S}_{k}$ is linearly independent. For $1\le i\le k1$, it follows from (1) that
since $\langle {v}_{j},\text{}{v}_{i}\rangle =0$ if $i\ne j$ by the induction assumption that ${S}_{k1}^{\prime}$ is orthogonal. Hence ${S}_{k}^{\prime}$ is an orthogonal set of nonzero vectors. Now, by (1), we have that $\text{span}({S}_{k}^{\prime})\subseteq \text{span}({S}_{k})$. But by Corollary 2 to Theorem 6.3, ${S}_{k}^{\prime}$ is linearly independent; so $\mathrm{dim}(\text{span}({S}_{k}^{\prime}))=\mathrm{dim}(\text{span}({S}_{k}))=k$. Therefore $\text{span}({S}_{k}^{\prime})=\text{span}({S}_{k})$.
The construction of $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ by the use of Theorem 6.4 is called the Gram—Schmidt process.
Example 4
In ${\text{R}}^{4}$, let ${w}_{1}=(1,\text{}0,\text{}1,\text{}0),\text{}{w}_{2}=(1,\text{}1,\text{}1,\text{}1),$ and ${w}_{3}=(0,\text{}1,\text{}2,\text{}1)$. Then $\{{w}_{1},\text{}{w}_{2},\text{}{w}_{3}\}$ is linearly independent. We use the GramSchmidt process to compute the orthogonal vectors ${v}_{1},\text{}{v}_{2},$ and ${v}_{3}$, and then we normalize these vectors to obtain an orthonormal set.
Take ${v}_{1}={w}_{1}=(1,\text{}0,\text{}1,\text{}0)$. Then
Finally,
These vectors can be normalized to obtain the orthonormal basis $\{{u}_{1},\text{}{u}_{2},\text{}{u}_{3}\}$, where
and
Example 5
Let $\text{V}=\text{P}(R)$ with the inner product $\langle f(x),\text{}g(x)\rangle ={\int}_{1}^{1}f(t)g(t)\text{}dt$, and consider the subspace ${\text{P}}_{2}(R)$ with the standard ordered basis $\beta $. We use the GramSchmidt process to replace $\beta $ by an orthogonal basis $\{{v}_{1},\text{}{v}_{2},\text{}{v}_{3}\}$ for ${\text{P}}_{2}(R)$, and then use this orthogonal basis to obtain an orthonormal basis for ${\text{P}}_{2}(R)$.
Take ${v}_{1}=1$. Then ${v}_{1}{}^{2}={\displaystyle {\int}_{1}^{1}{1}^{2}\text{}dt=2}$, and $\langle x,\text{}{v}_{1}\rangle ={\displaystyle {\int}_{1}^{1}t\cdot 1\text{}dt=0.}$. Thus
Furthermore,
Therefore
We conclude that $\left\{1,\text{}x,\text{}{x}^{2}{\displaystyle \frac{1}{3}}\right\}$ is an orthogonal basis for ${\text{P}}_{2}(R)$.
To obtain an orthonormal basis, we normalize ${v}_{1},\text{}{v}_{2},$, and ${v}_{3}$ to obtain
and similarly,
Thus $\{{u}_{1},\text{}{u}_{2},\text{}{u}_{3}\}$ is the desired orthonormal basis for ${\text{P}}_{2}(R)$.
Continuing to apply the GramSchmidt orthogonalization process to the basis $\{1,\text{}x,\text{}{x}^{2},\text{}\dots \}$ for P(R), we obtain an orthogonal basis $\{{v}_{1},\text{}{v}_{2},\text{}{v}_{3}\text{}\dots \}$. For each n, the polynomial $(1/{v}_{k}(1)){v}_{k}$ is called the kth Legendre polynomial. The first three Legendre polynomials are 1, x and $\frac{1}{2}}(3{x}^{2}1)$. The set of Legendre polynomials is also an orthogonal basis for P(R).
The following result gives us a simple method of representing a vector as a linear combination of the vectors in an orthonormal basis.
Theorem 6.5.
Let V be a nonzero finitedimensional inner product space. Then V has an orthonormal basis $\beta $. Furthermore, if $\beta =\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ and $x\in \text{V}$, then
Proof.
Let ${\beta}_{0}$ be an ordered basis for V. Apply Theorem 6.4 to obtain an orthogonal set ${\beta}^{\prime}$ of nonzero vectors with $\text{span}({\beta}^{\prime})=\text{span}({\beta}_{0})=\text{V}$. By normalizing each vector in ${\beta}^{\prime}$, we obtain an orthonormal set $\beta $ that generates V. By Corollary 2 to Theorem 6.3, $\beta $ is linearly independent; therefore $\beta $ is an orthonormal basis for V. The remainder of the theorem follows from Corollary 1 to Theorem 6.3.
Example 6
We use Theorem 6.5 to represent the polynomial $f(x)=1+2x+3{x}^{2}$ as a linear combination of the vectors in the orthonormal basis $\{{u}_{1},\text{}{u}_{2},\text{}{u}_{3}\}$ for ${\text{P}}_{2}(R)$ obtained in Example 5. Observe that
and
Therefore $f(x)=2\sqrt{2}{u}_{1}+{\displaystyle \frac{2\sqrt{6}}{3}}{u}_{2}+{\displaystyle \frac{2\sqrt{10}}{5}}{u}_{3}$.
Theorem 6.5 gives us a simple method for computing the entries of the matrix representation of a linear operator with respect to an orthonormal basis.
Corollary.
Let V be a finitedimensional inner product space with an orthonormal basis $\beta =\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$. Let T be a linear operator on V, and let $A={[\text{T}]}_{\beta}$. Then for any i and j, ${A}_{ij}=\langle T({v}_{j}),\text{}{v}_{i}\rangle $.
Proof. From Theorem 6.5, we have
Hence ${A}_{ij}=\langle \text{T}({v}_{j}),\text{}{v}_{i}\rangle $.
The scalars $\langle x,\text{}{v}_{i}\rangle $ given in Theorem 6.5 have been studied extensively for special inner product spaces. Although the vectors ${v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}$ were chosen from an orthonormal basis, we introduce a terminology associated with orthonormal sets $\beta $ in more general inner product spaces.
Definition.
Let $\beta $ be an orthonormal subset (possibly infinite) of an inner product space V, and let $x\in \text{V}$. We define the Fourier coefficients of x relative to $\beta $ to be the scalars $\langle x,\text{}y\rangle $, where $y\in \beta $.
In the first half of the 19th century, the French mathematician Jean Baptiste Fourier was associated with the study of the scalars
or in the complex case,
for a function f. In the context of Example 9 of Section 6.1, we see that ${c}_{n}=\langle f,\text{}{f}_{n}\rangle $, where ${f}_{n}(t)={e}^{int};$ that is, ${c}_{n}$ is the nth Fourier coefficient for a continuous function $f\in \text{V}$ relative to S. The coefficients ${c}_{n}$ are the “classical” Fourier coefficients of a function, and the literature concerning their behavior is extensive. We learn more about Fourier coefficients in the remainder of this chapter.
Example 7
Let $S=\{{e}^{int}:\text{}n\text{is an integer}\}$. In Example 9 of Section 6.1, S was shown to be an orthonormal set in H. We compute the Fourier coefficients of $f(t)=t$ relative to S. Using integration by parts, we have, for $n\ne 0$,
and, for $n=0$,
As a result of these computations, and using Exercise 16 of this section, we obtain an upper bound for the sum of a special infinite series as follows:
for every k. Now, using the fact that $f{}^{2}={\displaystyle \frac{4}{3}}{\pi}^{2}$, we obtain
or
Because this inequality holds for all k, we may let $k\to \infty $ to obtain
Additional results may be produced by replacing f by other functions.
We are now ready to proceed with the concept of an orthogonal complement.
Definition.
Let S be a nonempty subset of an inner product space V. We define ${S}^{\perp}$ (read “S perp”) to be the set of all vectors in V that are orthogonal to every vector in S; that is, ${S}^{\perp}=\{x\in \text{V}:\langle x,\text{}y\rangle =0\text{for all}y\in S\}$. The set ${S}^{\perp}$ is called the orthogonal complement of S.
It is easily seen that ${S}^{\perp}$ is a subspace of V for any subset S of V.
Example 8
The reader should verify that ${\left\{0\right\}}^{\perp}=\text{V}$ and ${\text{V}}^{\perp}=\left\{0\right\}$ for any inner product space V.
Example 9
If $\text{V}={\text{R}}^{3}$ and $S=\left\{{e}_{3}\right\}$, then ${S}^{\perp}$ equals the xyplane (see Exercise 5).
Exercise 18 provides an interesting example of an orthogonal complement in an infinitedimensional inner product space.
Consider the problem in ${\text{R}}^{3}$ of finding the distance from a point P to a plane W. (See Figure 6.2.) Problems of this type arise in many settings. If we let y be the vector determined by 0 and P, we may restate the problem as follows: Determine the vector u in W that is “closest” to y. The desired distance is clearly given by $yu$. Notice from the figure that the vector $z=yu$ is orthogonal to every vector in W, and so $z\in {\text{W}}^{\perp}$.
The next result presents a practical method of finding u in the case that W is a finitedimensional subspace of an inner product space.
Theorem 6.6.
Let W be a finitedimensional subspace of an inner product space V, and let $y\in \text{V}$. Then there exist unique vectors $u\in \text{W}$ and $z\in {\text{W}}^{\perp}$ such that $y=u+z$. Furthermore, if $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$ is an orthonormal basis for W, then
Proof.
Let $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$ be an orthonormal basis for W, let u be as defined in the preceding equation, and let $z=yu$. Clearly $u\in \text{W}$ and $y=u+z$.
To show that $z\in {\text{W}}^{\perp}$, it suffices to show, by Exercise 7, that z is orthogonal to each ${v}_{j}$. For any j, we have
To show uniqueness of u and z, suppose that $y=u+z={u}^{\prime}+{z}^{\prime}$, where ${u}^{\prime}\in \text{W}$ and ${z}^{\prime}\in {\text{W}}^{\perp}$. Then $u{u}^{\prime}={z}^{\prime}z\in \text{W}\cap {\text{W}}^{\perp}=\left\{0\right\}$. Therefore $u{u}^{\prime}$ and $z={z}^{\prime}$.
Corollary.
In the notation of Theorem 6.6, the vector u is the unique vector in W that is “closest” to y; that is, for any $x\in \text{W},yx\ge yu$, and this inequality is an equality if and only if $x=u$.
Proof.
As in Theorem 6.6, we have that $y=u+z$, where $z\in {\text{W}}^{\perp}$. Let $x\in \text{W}$. Then $ux$ is orthogonal to z, so, by Exercise 10 of Section 6.1, we have
Now suppose that $yx=yu$. Then the inequality above becomes an equality, and therefore $ux{}^{2}+z{}^{2}=z{}^{2}$. It follows that $ux=0$, and hence $x=u$. The proof of the converse is obvious.
The vector u in the corollary is called the orthogonal projection of y on W. We will see the importance of orthogonal projections of vectors in the application to least squares in Section 6.3.
Example 10
Let $\text{V}={\text{P}}_{3}(R)$ with the inner product
We compute the orthogonal projection ${f}_{1}(x)$ of $f(x)={x}^{3}$ on ${\text{P}}_{2}(R)$.
By Example 5,
is an orthonormal basis for ${\text{P}}_{2}(R)$. For these vectors, we have
and
Hence
It was shown (Corollary 2 to the replacement theorem, p. 48) that any linearly independent set in a finitedimensional vector space can be extended to a basis. The next theorem provides an interesting analog for an orthonormal subset of a finitedimensional inner product space.
Theorem 6.7.
Suppose that $S=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$ is an orthonormal set in an ndimensional inner product space V. Then

(a) S can be extended to an orthonormal basis $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k},\text{}{v}_{k+1},\text{}\dots ,\text{}{v}_{n}\}$ for V.

(b) If $\text{W}=\text{span}(S)$, then ${S}_{1}=\{{v}_{k+1},\text{}{v}_{k+2},\text{}\dots ,\text{}{v}_{n}\}$ is an orthonormal basis for ${\text{W}}^{\perp}$ (using the preceding notation).

(c) If W is any subspace of V, then $\mathrm{dim}(\text{V})=\mathrm{dim}(\text{W})+\mathrm{dim}({\text{W}}^{\perp})$.
Proof.
(a) By Corollary 2 to the replacement theorem (p. 48), S can be extended to an ordered basis ${S}^{\prime}=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k},\text{}{w}_{k+1},\text{}\dots ,\text{}{w}_{n}\}$ for V. Now apply the GramSchmidt process to ${S}^{\prime}$. The first k vectors resulting from this process are the vectors in S by Exercise 8, and this new set spans V. Normalizing the last $nk$ vectors of this set produces an orthonormal set that spans V. The result now follows.
(b) Because ${S}_{1}$ is a subset of a basis, it is linearly independent. Since ${S}_{1}$ is clearly a subset of ${\text{W}}^{\perp}$, we need only show that it spans ${\text{W}}^{\perp}$. Note that, for any $x\in \text{V}$, we have
If $x\in {\text{W}}^{\perp}$, then $\langle x,\text{}{v}_{i}\rangle =0$ for $1\le i\le k$. Therefore
(c) Let W be a subspace of V. It is a finitedimensional inner product space because V is, and so it has an orthonormal basis $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{k}\}$. By (a) and (b), we have
Example 11
Let $\text{W}=\text{span}(\{{e}_{1},\text{}{e}_{2}\})$ in ${\text{F}}^{3}$. Then $x=(a,\text{}b,\text{}c)\in {\text{W}}^{\perp}$ if and only if $0=\langle x,\text{}{e}_{1}\rangle =a$ and $0=\langle x,\text{}{e}_{2}\rangle =b$. So $x=(0,\text{}0,\text{}c)$, and therefore ${\text{W}}^{\perp}={\text{span({e}}_{3}\text{})}$. One can deduce the same result by noting that ${e}_{3}\in {\text{W}}^{\perp}$ and, from (c), that $\mathrm{dim}({\text{W}}^{\perp})=32=1$.
Exercises

Label the following statements as true or false.

(a) The GramSchmidt orthogonalization process produces an orthonormal set from an arbitrary linearly independent set.

(b) Every nonzero finitedimensional inner product space has an orthonormal basis.

(c) The orthogonal complement of any set is a subspace.

(d) If $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ is a basis for an inner product space V, then for any $x\in \text{V}$ the scalars $\langle x,\text{}{v}_{i}\rangle $ are the Fourier coefficients of x.

(e) An orthonormal basis must be an ordered basis.

(f) Every orthogonal set is linearly independent.

(g) Every orthonormal set is linearly independent.


In each part, apply the Gram–Schmidt process to the given subset S of the inner product space V to obtain an orthogonal basis for span(S). Then normalize the vectors in this basis to obtain an orthonormal basis $\beta $ for span(S), and compute the Fourier coefficients of the given vector relative to $\beta $. Finally, use Theorem 6.5 to verify your result.

(a) $\text{V}={\text{R}}^{3},\text{}S=\{(1,\text{}0,\text{}1),\text{}(0,\text{}1,\text{}1),\text{}(1,\text{}3,\text{}3)\},$ and $x=(1,\text{}1,\text{}2)$

(b) $\text{V}={\text{R}}^{3},\text{}S=\{(1,\text{}1,\text{}1),\text{}(0,\text{}1,\text{}1),\text{}(0,\text{}0,\text{}1)\},$ and $x=(1,\text{}0,\text{}1)$

(c) $\text{V}={\text{P}}_{2}(R)$ with the inner product $\langle f(x),\text{}g(x)\rangle ={\displaystyle {\int}_{0}^{1}f(t)g(t)\text{}dt,}\text{}S=\{1,\text{}x,\text{}{x}^{2}\}$, and $h(x)=1+x$

(d) $\text{V}=\text{span}(S)$, where $S=\{(1,\text{}i,\text{}0),\text{}(1i,\text{}2,\text{}4i)\}$ and $x=(3+i,\text{}4i,\text{}4)$

(e) $\text{V}={\text{R}}^{4},\text{}S=\{(2,\text{}1,\text{}2,\text{}4),\text{}(2,\text{}1,\text{}5,\text{}5),\text{}(1,\text{}3,\text{}7,\text{}11)\},$ and $x=(11,\text{}8,\text{}4,\text{}18)$

(f) $\text{V}={\text{R}}^{4},\text{}S=\{(1,\text{}2,\text{}1,\text{}3),\text{}(3,\text{}6,\text{}3,\text{}1),\text{}(1,\text{}4,\text{}2,\text{}8)\},$ and $x=(1,\text{}2,\text{}1,\text{}1)$

(g) $\text{V}={\text{M}}_{2\times 2}(R),\text{}S=\left\{\left(\begin{array}{rr}3& 5\\ 1& 1\end{array}\right),\text{}\left(\begin{array}{rr}1& 9\\ 5& 1\end{array}\right),\text{}\left(\begin{array}{rr}7& 17\\ 2& 6\end{array}\right)\right\},$ and $A=\left(\begin{array}{rr}1& 27\\ 4& 8\end{array}\right)$

(h) $\text{V}={\text{M}}_{2\times 2}(R),\text{}S=\left\{\left(\begin{array}{rr}2& 2\\ 2& 1\end{array}\right),\text{}\left(\begin{array}{rr}11& 4\\ 2& 5\end{array}\right),\text{}\left(\begin{array}{rr}4& 12\\ 3& 16\end{array}\right)\right\}$, and $A=\left(\begin{array}{rr}8& 6\\ 25& 13\end{array}\right)$

(i) $\text{V}=\text{span}(S)$ with the inner product $\langle f,\text{}g\rangle ={\displaystyle {\int}_{0}^{\pi}f(t)g(t)\text{}dt,}\text{}S=\{\mathrm{sin}\text{}t,\text{}\mathrm{cos}\text{}t,\text{}1,\text{}t\},$ and $h(t)=2t+1$

(j) $\text{V}={\text{C}}^{4},\text{}S=\{(1,\text{}i,\text{}2i,\text{}1),\text{}(2+3i,\text{}3i,\text{}1i,\text{}2i),\text{}(1+7i,\text{}6+10i,\text{}114i,\text{}3+4i)\},$ and $x=(2+7i,\text{}6+9i,\text{}93i,\text{}4+4i)$

(k) $\text{V}={\text{C}}^{4},\text{}S=\{(4,\text{}32i,\text{}i,\text{}14i),\text{}(15i,\text{}54i,\text{}3+5i,\text{}72i),\text{}(27i,\text{}76i,\text{}15+25i,\text{}76i)\},$ and $x=(137i,\text{}12+3i,\text{}3911i,\text{}26+5i)$

(l) $\text{V}={\text{M}}_{2\times 2}(C),\text{}S=\left\{\left(\begin{array}{rr}1i& 23i\\ 2+2i& 4+i\end{array}\right),\text{}\left(\begin{array}{rr}8i& 4\\ 33i& 4+4i\end{array}\right),\text{}\left(\begin{array}{rr}2538i& 213i\\ 1278i& 7+24i\end{array}\right)\right\},$ and $A=\left(\begin{array}{rr}2+8i& 13+i\\ 1010i& 99i\end{array}\right)$

(m) $\text{V}={\text{M}}_{2\times 2}(C),\text{}S=\left\{\left(\begin{array}{rr}1+i& i\\ 2i& 1+3i\end{array}\right),\text{}\left(\begin{array}{rr}17i& 98i\\ 1+10i& 62i\end{array}\right),\text{}\left(\begin{array}{rr}11132i& 3431i\\ 7126i& 715i\end{array}\right)\right\},$ and $A=\left(\begin{array}{rr}7+5i& 3+18i\\ 96i& 3+7i\end{array}\right)$


In ${\text{R}}^{2}$, let
$$\beta =\left\{\left({\displaystyle \frac{1}{\sqrt{2}}},\text{}{\displaystyle \frac{1}{\sqrt{2}}}\right),\text{}\left({\displaystyle \frac{1}{\sqrt{2}}},\text{}{\displaystyle \frac{1}{\sqrt{2}}}\right)\right\}.$$Find the Fourier coefficients of (3, 4) relative to $\beta $ .

Let $S=\{(1,\text{}0,\text{}i),\text{}(1,\text{}2,\text{}1)\}$ in ${\text{C}}^{3}$. Compute ${S}^{\perp}$.

Let ${S}_{0}=\left\{{x}_{0}\right\}$, where ${x}_{0}$ is a nonzero vector in ${\text{R}}^{3}$. Describe ${S}_{0}^{\perp}$ geometrically. Now suppose that $S=\{{x}_{1},\text{}{x}_{2}\}$ is a linearly independent subset of ${\text{R}}^{3}$. Describe ${S}^{\perp}$ geometrically.

Let V be an inner product space, and let W be a finitedimensional subspace of V. If $x\notin \text{W}$, prove that there exists $y\in \text{V}$ such that $y\in {\text{W}}^{\perp}$, but $\langle x,\text{}y\rangle \ne 0$. Hint: Use Theorem 6.6.

Let $\beta $ be a basis for a subspace W of an inner product space V, and let $z\in \text{V}$. Prove that $z\in {\text{W}}^{\perp}$ if and only if $\langle z,\text{}v\rangle =0$ for every $v\in \beta $.

Prove that if $\{{w}_{1},\text{}{w}_{2},\text{}\dots ,\text{}{w}_{n}\}$ is an orthogonal set of nonzero vectors, then the vectors ${v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}$ derived from the GramSchmidt process satisfy ${v}_{i}={w}_{i}$ for $i=1,\text{\hspace{0.17em}}2,\text{\hspace{0.17em}}\dots ,\text{\hspace{0.17em}}n$. Hint: Use mathematical induction.

Let $\text{W}=\text{span}(\{(i,\text{}0,\text{}1)\})$ in ${\text{C}}^{3}$. Find orthonormal bases for W and ${\text{W}}^{\perp}$.

Let W be a finitedimensional subspace of an inner product space V. Prove that $\text{V}=\text{W}\oplus {\text{W}}^{\perp}$. Using the definition on page 76, prove that there exists a projection T on W along ${\text{W}}^{\perp}$ that satisfies $\text{N}(\text{T})={\text{W}}^{\perp}$. In addition, prove that $\text{T}(x)\le x$ for all $x\in \text{V}$. Hint: Use Theorem 6.6 and Exercise 10 of Section 6.1.

Let A be an $n\times n$ matrix with complex entries. Prove that $AA\text{*}=I$ if and only if the rows of A form an orthonormal basis for ${\text{C}}^{n}$. Visit goo.gl/
iKcC4S for a solution. 
Prove that for any matrix $A\in {\text{M}}_{m\times n}(F),\text{}{(\text{R}({\text{L}}_{A\text{*}}))}^{\perp}=\text{N}({\text{L}}_{A})$.

Let V be an inner product space, S and ${S}_{0}$ be subsets of V, and W be a finitedimensional subspace of V. Prove the following results.

(a) ${S}_{0}\subseteq S$ implies that ${S}^{\perp}\subseteq {S}_{0}^{\perp}$.

(b) $S\subseteq {({S}^{\perp})}^{\perp};$ so $\text{span}(S)\subseteq {({S}^{\perp})}^{\perp}$.

(c) $\text{W}={({\text{W}}^{\perp})}^{\perp}$. Hint: Use Exercise 6.

(d) $\text{V}=\text{W}\oplus {\text{W}}^{\perp}$. (See the exercises of Section 1.3.)


Let ${\text{W}}_{1}$ and ${\text{W}}_{2}$ be subspaces of a finitedimensional inner product space. Prove that ${({\text{W}}_{1}+{\text{W}}_{2})}^{\perp}={\text{W}}_{{}_{1}}^{\perp}\cap {\text{W}}_{2}^{\perp}$ and ${({\text{W}}_{1}\cap {\text{W}}_{2})}^{\perp}={\text{W}}_{{}_{1}}^{\perp}{\text{+W}}_{2}^{\perp}$. (See the definition of the sum of subsets of a vector space on page 22.) Hint for the second equation: Apply Exercise 13(c) to the first equation.

Let V be a finitedimensional inner product space over F.

(a) Parseval’s Identity. Let $\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ be an orthonormal basis for V. For any $x,\text{}y\in \text{V}$ prove that
$$\langle x,\text{}y\rangle ={\displaystyle \sum _{i=1}^{n}\langle x,\text{}{v}_{i}\rangle \overline{\langle y,\text{}{v}_{i}\rangle}.}$$ 
(b) Use (a) to prove that if $\beta $ is an orthonormal basis for V with inner product $\langle \cdot ,\text{}\cdot \rangle $, then for any $x,\text{}y\in \text{V}$
$${\langle {\varphi}_{\beta}(x),\text{}{\varphi}_{\beta}(y)\rangle}^{\prime}={\langle {[x]}_{\beta},\text{}{[y]}_{\beta}\rangle}^{\prime}=\langle x,\text{}y\rangle ,$$where ${\langle \cdot ,\text{}\cdot \rangle}^{\prime}$ is the standard inner product on ${\text{F}}^{n}$.



(a) Bessel’s Inequality. Let V be an inner product space, and let $S=\{{v}_{1},\text{}{v}_{2},\text{}\dots ,\text{}{v}_{n}\}$ be an orthonormal subset of V. Prove that for any $x\in \text{V}$ we have
$$x{}^{2}\ge {\displaystyle \sum _{i=1}^{n}\langle x,\text{}{v}_{i}\rangle {}^{2}}.$$Hint: Apply Theorem 6.6 to $x\in \text{V}$ and $\text{W}=\text{span}(S)$. Then use Exercise 10 of Section 6.1.

(b) In the context of (a), prove that Bessel’s inequality is an equality if and only if $x\in \text{span}(S)$.


Let T be a linear operator on an inner product space V. If $\langle \text{T}(x),\text{}y\rangle =0$ for all $x,\text{}y\in \text{V}$, prove that $\text{T}={\text{T}}_{0}$. In fact, prove this result if the equality holds for all x and y in some basis for V.

Let $\text{V}=\text{C}([1,\text{}1])$. Suppose that ${\text{W}}_{e}$ and ${\text{W}}_{o}$ denote the subspaces of V consisting of the even and odd functions, respectively. (See Exercise 22 of Section 1.3.) Prove that ${\text{W}}_{e}^{\perp}={\text{W}}_{o}$, where the inner product on V is defined by
$$\langle f,\text{}g\rangle ={\displaystyle {\int}_{1}^{1}f(t)g(t)\text{}dt.}$$ 
In each of the following parts, find the orthogonal projection of the given vector on the given subspace W of the inner product space V.

(a) $\text{V}={\text{R}}^{2},\text{}u=(2,\text{}6),$ and $\text{W}=\{(x,\text{}y):\text{}y=4x\}$

(b) $\text{V}={\text{R}}^{3},\text{}u=(2,\text{}1,\text{}3)$, and $\text{W}=\{(x,\text{}y,\text{}z):\text{}x+3y2z=0\}$

(c) $\text{V}=\text{P}(R)$ with the inner product $\langle f(x),\text{}g(x)\rangle ={\displaystyle {\int}_{0}^{1}f(t)g(t)\text{}dt},\text{}h(x)=4+3x2{x}^{2},$ and $\text{W}={\text{P}}_{1}(R)$


In each part of Exercise 19, find the distance from the given vector to the subspace W.

Let $\text{V}=\text{C}([1,\text{}1])$ with the inner product $\langle f,\text{}g\rangle ={\displaystyle {\int}_{1}^{1}f(t)g(t)\text{}dt}$, and let W be the subspace ${\text{P}}_{2}(R)$, viewed as a space of functions. Use the orthonormal basis obtained in Example 5 to compute the “best” (closest) seconddegree polynomial approximation of the function $h(t)={e}^{t}$ on the interval $[1,\text{}1]$.

Let $\text{V}=\text{C}([0,\text{}1])$ with the inner product $\langle f,\text{}g\rangle ={\int}_{0}^{1}f(t)g(t)\text{}dt$. Let W be the subspace spanned by the linearly independent set $\{t,\text{}\sqrt{t}\}$.

(a) Find an orthonormal basis for W.

(b) Let $h(t)={t}^{2}$. Use the orthonormal basis obtained in (a) to obtain the “best” (closest) approximation of h in W.


Let V be the vector space defined in Example 5 of Section 1.2, the space of all sequences $\sigma $ in F (where $F=R$ or $F=C$) such that $\sigma (n)\ne 0$ for only finitely many positive integers n. For $\sigma ,\text{}\mu \in \text{V}$, we define $\langle \sigma ,\text{}\mu \rangle ={\displaystyle \sum _{n=1}^{\infty}\sigma (n)\overline{\mu (n)}}$. Since all but a finite number of terms of the series are zero, the series converges.

(a) Prove that $\langle \cdot ,\text{}\cdot \rangle $ is an inner product on V, and hence V is an inner product space.

(b) For each positive integer n, let ${e}_{n}$ be the sequence defined by ${e}_{n}(k)={\delta}_{nk}$, where ${\delta}_{nk}$ is the Kronecker delta. Prove that $\{{e}_{1},\text{}{e}_{2},\text{}\dots \}$ is an orthonormal basis for V.

(c) Let ${\sigma}_{n}={e}_{1}+{e}_{n}$ and $\text{W}=\text{span}(\{{\sigma}_{n}:\text{}n\ge 2\}$.

(i) Prove that ${e}_{1}\notin \text{W}$, so $\text{W}\ne \text{V}$.

(ii) Prove that ${\text{W}}^{\perp}=\left\{0\right\}$, and conclude that $\text{W}\ne {({\text{W}}^{\perp})}^{\perp}$.
Thus the assumption in Exercise 13(c) that W is finitedimensional is essential.

