Class 03/19/21

Gram Schmidt process

You can find orthogonal basis from any basis ${v_{1}, v_{2}, ..., n}$

If you define ${w_{1}, w_{2}, ..., w_{n}}$

w_{1} = v_{1} w_{2} = v_{2} - p ro j_{w_{1}} v_{2} w_{3} = v_{3} - p ro j_{w_{1}} v_{3} - p ro j_{w_{2}} v_{3}

Where a projection is:

(\frac{v \cdot w _{n}}{w _{n} \cdot w _{n}})

Suppose $b$ is not in the $CO L (A)$ , $A x = b$ has no solution.

However, the projection: $A x = p ro j_{co l (A)} b$ is called a least-squares solution of $A x = b$ .

This is inherently the projection onto a space, and then finding the vector that is closest to that vector outside of the column space.

$∣∣ A x - b ∣ ∣^{2} < ∣∣ A x - b ∣ ∣^{2}$

$A x = p ro j_{co l (A)} b$

$b - A x$ is in the $CO L (A)^{⊥}$

$b - A x$ is in $N ULL (A^{T})$

$A^{T} (b - A x) = 0$

$A^{T} b - A^{T} A x = 0$

Normal equations: $A^{T} A x = A^{T} b$

Find a least squares solution of

⎣ ⎡ 1121 - 1 101 ⎦ ⎤ x = ⎣ ⎡ 12 - 1 1 ⎦ ⎤

Goes to:

[7113] [x_{1} x_{2}] = [22]

Ans:

\frac{1}{20} [12]

Find $p ro j_{co l_{A}} b$

Multiply it by A:

⎣ ⎡ - 2 474 ⎦ ⎤

$∣∣ b - A x ∣ ∣^{2}$

The length of the projection vector is error amount.