Orthogonal Projection

Section 6.2 Orthogonal Projection

Subsection 6.2.1 Orthogonal and orthonormal set

Definitions of orthogonal and orthonormal.

A set \(S=\{v_1,v_2,\ldots,v_m\}\) of vectors in \(\mathbb{R}^{n}\) is orthogonal when every pair of vectors in \(S\) is orthogonal. That is, \(v_i\cdot v_j=0\) for all \(1\leq i\neq j\leq m\text{.}\)

If, in addition, each vector in the set is a unit vector, then \(S\) is orthonormal.

Example 6.2.1.

Show that the set \(S\) below is an orthogonal set of \(\mathbb{R}^{4}\text{.}\)

\begin{equation*} S=\{(2,3,2,-2),(1,0,0,1),(-1,0,2,1),(-1,2,-1,1)\} \end{equation*}

One can check \(S\) is orthogonal for every oair of vectors. You may answer this by matrix multiplication.

In next section, we turn a linear independent set into an orthonormal set. The method is called Gram-Schmidt Process

Subsection 6.2.2 Orthogonal Projection

Let \(\mathbf{u}\) and \(\mathbf{v}\) be vectors in \(\mathbb{R}^{n}\) such that \(\mathbf{v} \neq \mathbf{0}\text{.}\) Then the orthogonal projection of \(\mathbf{u}\) onto \(\mathbf{v}\) is the vector \(k\mathbf{v}\) (\(k\in \mathbb{R}\)) such that \(\|\mathbf{u}-k\mathbf{v}\|\) attains minimum, denoted by \(\operatorname{proj}_{\mathbf{v}} \mathbf{u}\text{.}\)

Theorem 6.2.2.

Let \(\mathbf{u}\) and \(\mathbf{v}\) be vectors in \(\mathbb{R}^{n}\text{,}\) such that \(\mathbf{v} \neq \mathbf{0}\text{.}\) Then the orthogonal projection of \(\mathbf{u}\) onto \(\mathbf{v}\) is

\begin{equation*} \operatorname{proj}_{\mathbf{v}} \mathbf{u}=\frac{\mathbf{u}\cdot \mathbf{v}}{\mathbf{v}\cdot\mathbf{v}} \mathbf{v}. \end{equation*}

Figure 6.2.3. Given a nonzero vector \(\mathbf{w}\) and a vector \(\mathbf{b}\text{,}\) the orthogonal projection of vector \(\mathbf{b}\) on \(w\) is the vector \(\widehat{\mathbf{b}}\text{.}\)

Discussion: Describe the relation between the vector \(\mathbf{b}-\widehat{\mathbf{b}}\) and the vector \(\mathbf{w}\text{.}\)

Proposition 6.2.4. Projection formula.

If \(W\) is a subspace of \(\mathbb{R}^{n}\) having an orthogonal basis \(v_1,v_2,\ldots, v_m\) and \(w\) is a vector in \(\mathbb{R}^{n}\text{,}\) then the orthogonal projection \(\hat{w}\) of \(w\) onto \(W\) is

\begin{equation*} \hat{w}=\frac{w\cdot v_1}{v_1\cdot v_1}~v_1 + \frac{w\cdot v_2}{v_2\cdot v_2}~v_2 + \cdots + \frac{w\cdot v_m}{v_m\cdot v_m}~v_m \end{equation*}

There is a great observation if the basis is orthonormal.

Proposition 6.2.5.

If \(W\) is a subspace of \(\mathbb{R}^{n}\) having an orthonormal basis \(v_1,v_2,\ldots, v_m\) and \(w\) is a vector in \(\mathbb{R}^{n}\text{,}\) then the orthogonal projection \(\hat{w}\) of \(w\) onto \(W\) is

\begin{equation*} \hat{w}=\left(QQ^{T}\right)(w), \end{equation*}

where \(Q=[v_1\, v_2\, \ldots\, v_m].\)

Proof.

It follows from Proposition 6.2.4 that

\begin{align*} \hat{w} \amp= (w\cdot v_1)~v_1 + (w\cdot v_2)~v_2 + \cdots + (w\cdot v_m)~v_m\\ \amp = [v_1\, v_2\, \ldots\, v_m]\left[\begin{array}{c} w\cdot v_1\\ w\cdot v_2\\ \ddots \\ w\cdot v_m \end{array}\right]\\ \amp = [v_1\, v_2\, \ldots\, v_m]\left[\begin{array}{c} v_1\cdot w\\ v_2\cdot w\\ \ddots \\ v_m\cdot w \end{array}\right]\\ \amp = [v_1\, v_2\, \ldots\, v_m]\left[\begin{array}{c} v_1^{T}w\\ v_2^{T}w\\ \ddots \\ v_m^{T} w \end{array}\right]\\ \amp = [v_1\, v_2\, \ldots\, v_m]\left[\begin{array}{c} v_1^{T}\\ v_2^{T}\\ \ddots \\ v_m^{T} \end{array}\right]w\\ \amp=\left(QQ^{T}\right)(w) \end{align*}

Prev Top Next